This invention relates to audio processing systems. More particularly, the invention relates to kernel-mode audio processing modules.
Musical performances have become a key component of electronic and multimedia products such as stand-alone video game devices, computer-based video games, computer-based slide show presentations, computer animation, and other similar products and applications. As a result, music generating devices and music playback devices are now tightly integrated into electronic and multimedia components.
Musical accompaniment for multimedia products can be provided in the form of digitized audio streams. While this format allows recording and accurate reproduction of non-synthesized sounds, it consumes a substantial amount of memory. As a result, the variety of music that can be provided using this approach is limited. Another disadvantage of this approach is that the stored music cannot be easily varied. For example, it is generally not possible to change a particular musical part, such as a bass part, without re-recording the entire musical stream.
Because of these disadvantages, it has become quite common to generate music based on a variety of data other than pre-recorded digital streams. For example, a particular musical piece might be represented as a sequence of discrete notes and other events corresponding generally to actions that might be performed by a keyboardistxe2x80x94such as pressing or releasing a key, pressing or releasing a sustain pedal, activating a pitch bend wheel, changing a volume level, changing a preset, etc. An event such as a note event is represented by some type of data structure that includes information about the note such as pitch, duration, volume, and timing. Music events such as these are typically stored in a sequence that roughly corresponds to the order in which the events occur. Rendering software retrieves each music event and examines it for relevant information such as timing information and information relating the particular device or xe2x80x9cinstrumentxe2x80x9d to which the music event applies. The rendering software then sends the music event to the appropriate device at the proper time, where it is rendered. The MIDI (Musical Instrument Digital Interface) standard is an example of a music generation standard or technique of this type, which represents a musical performance as a series of events.
Computing devices, such as many modern computer systems, allow MIDI data to be manipulated and/or rendered. These computing devices are frequently built based on an architecture employing multiple privilege levels, often referred to as user-mode and kernel-mode. Manipulation of the MIDI data is typically performed by one or more applications executing in user-mode, while the input of data from and output of data to hardware is typically managed by an operating system or a driver executing in kernel-mode.
Such a setup requires the MIDI data to be received by the driver or operating system executing in kernel-mode, transferred to the application executing in user-mode, manipulated by the application as needed in user-mode, and then transferred back to the operating system or driver executing in kernel-mode for rendering. Data transfers between kernel-mode and user-mode, however, can take a considerable and unpredictable amount of time. Lengthy delays can result in unacceptable latency, particularly for real-time audio playback, while unpredictability can result in an unacceptable amount of jitter in the audio data, resulting in unacceptable rendering of the audio data.
The invention described below addresses these disadvantages, providing kernel-mode audio processing modules.
Kernel-mode audio processing modules are described herein.
According to one aspect, multiple audio processing modules or filters are combined to form a module or filter graph. The graph is implemented in kernel-mode, reducing latency and jitter when handling audio data (e.g., MIDI data) by avoiding transfers of the audio data to user-mode applications for processing. A variety of different audio processing modules can be used to provide various pieces of functionality when processing audio data.
According to another aspect, a Feeder In filter is included to convert audio data received from a hardware driver into a data structure including a data portion that can include one of audio data, a pointer to a chain of additional data structures that include the audio data, and a pointer to a data buffer.
According to another aspect, a Feeder Out filter is included to convert, to a hardware driver-specific format, audio data received as part of a data structure including a data portion that can include one of audio data, a pointer to a chain of additional data structures that include the audio data, and a pointer to a data buffer.
According to another aspect, a Channel Group Mute filter is included to delete channel groups. Data packets corresponding to channel groups which match a filter parameter are forwarded to an allocator module for re-allocation of the memory space used by the data packets.
According to another aspect, a Channel Group Solo filter is included to delete all channel groups except for selected channel groups. Data packets corresponding to channel groups which do not match a filter parameter are forwarded to an allocator module for re-allocation of the memory space used by the data packets.
According to another aspect, a Channel Group Route filter is included to route groups of channels. The channel group identifiers for data packets corresponding to channel groups which match a filter parameter are changed to a new channel group.
According to another aspect, a Channel Group Map filter is included to alter channel group identifiers for multiple channel groups. The channel group identifiers for data packets corresponding to multiple source channel groups which match a filter parameter are changed to one or more different destination groups.
According to another aspect, a Channel Map filter to change any one or more of multiple channels to any one or more of the channels. Channels for data packets corresponding to multiple channels which match a filter parameter are changed to one or more different new channels. Additional data packets are generated as necessary in the event of multiple new channels (a one to many mapping of channels).
According to another aspect, a Message Filter is included to delete selected message types. Data packets corresponding to message types which match a filter parameter are forwarded to an allocator module for re-allocation of the memory space used by the data packets.
According to another aspect, a Note Map Curve filter is included to alter note values on an individual basis. An input note to output note mapping table is used to identify, for each received data packet, what the input note is to be changed to (if anything).
According to another aspect, a Velocity Map Curve filter is included to alter velocity values on an individual basis. An input velocity to output velocity mapping table is used to identify, for each received data packet, what the input velocity is to be changed to (if anything).
According to another aspect, a Note and Velocity Map Curve filter is included to allow combined note and velocity alterations based on both the input note and velocity valuesxe2x80x94two degrees of freedom, leading to much more expressive translations. A table mapping input note and velocity combinations to output note and velocity combinations is used to identify, for each received data packet, what the input note and velocity are to be changed to (if anything). Alternatively, rather than changing the input note and velocity values, the Note and Velocity Map Curve filter may generate a new data structure that includes the new note and velocity values (from the table), and then forward both on to the next module in the graph.
According to another aspect, a Time Palette filter is included to alter presentation times corresponding to the audio data. Presentation times can be quantized (e.g., snapped to a closest one of a set of presentation times) or anti-quantized (e.g., moved away from a set of presentation times). The presentation times can also be altered to generate a swing beat.
According to another aspect, a Variable Detune filter is included to alter the pitch of music by a variable offset value. The pitch of audio data corresponding to received data packets is altered by an amount that varies over time.
According to another aspect, an Echo filter is included to generate an echo for notes of the audio data. Additional data packets are generated that duplicate at least part of a received data packet, but increase the presentation time and decrease the velocity to generate an echo. The note values of the additional data packets may also be altered (e.g., for a spiraling up or spiraling down echo).
According to another aspect, a Profile System Performance filter is included to monitor and record system performance. System performance is monitored (e.g., a difference between presentation time for a data packet and the reference clock time just prior to rendering) and recorded for subsequent retrieval.